Return-oriented programming (ROP) is a computer security exploit technique in which an attacker uses software control of a stack to execute an attacker-chosen sequence of machine instructions. These clusters of instructions typically end with a programmer-intended or unintended return (RET) instruction within existing program code. The intended or unintended RET instruction transfers execution to the attacker-chosen return address on the stack and allows the attacker to retain execution control through the program code, and direct execution to the next set of chosen sequence of instructions to achieve the attacker's intent. The clusters of attacker-chosen instruction sequences are referred to as gadgets.
Often the executed gadget includes only several assembler instructions followed by a RET instruction that can already perform a well-defined attack operation. By chaining together a set of these gadgets such that the RET instructions from one gadget lands into the next gadget and so on, the malware writer is able to execute a complex algorithm without injecting any code into the program. Some of these instruction sequences ending in a RET can be found in functions compiled into the program or libraries.
Thus the ROP technique involves delivering a payload having a set of chained list of pointers to gadgets and parameters to a data memory of a program using vulnerabilities like stack buffer overflows. The exploit also overwrites the return address of the vulnerable function that was used to perform the stack buffer overflow to point to the first gadget in the sequence. When this vulnerable function executes a RET instruction, control transfers to the first gadget instead of the function caller. This gadget can then consume one or more data elements from the stack payload. Using this exploit type, the malware writer is able to change the control flow of the program by causing a control transfer to a non-programmer intended location in the program (e.g., to the middle of an instruction).
A ROP attack technique can use various characteristics of an x86 instruction set architecture (ISA): variable length and unaligned instruction encoding; large and dense ISA encoding; a stack holding control and data information; and a single byte opcode RET instruction. Current techniques to defend against such attacks can be ineffective and have various shortcomings.